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Section /. Introduction 



Goals and Overview of Sourcebook 

School and district administrators are called upon to make a multitude of decisions 
ranging from the mundane to the academic. Examples of some of the decisions that come 
up include: 

• Should we adopt this new program? 

• Are these test items measuring the curriculum? 

• Which textbooks should we choose? 

• How can we tell if our students are learning the material? 

• How many more bilingual teachers will we need next year? 

• How many computers can we buy? 

• Have changes in grouping practices affected the performance 
of higher achieving students? 

To help administrators make such decisions, they are often bombarded with data: 
inundated with reports, quality indicators, data, and statistics that describe the nature of 
their school or district. These include test scores, attendance rates, discipline records, and 
numerous others. And although massive changes in the availability of data and 
technology have been seen, there has been little change in the uses of data. Often 
decisions are made without the support of data that are hard and timely, and data that are 
available are often old and are seldom well-suited to the decision at hand. Administrators 
are sometimes at a loss as to how to make a coherent whole out of the different sources of 
data and how to integrate them with their knowledge about their school or district. 

While they may have a tendency to make decisions based on their professional 
knowledge, they also have a nagging feeling that somehow these statistical data are 
important and must be attended to, especially if they can be used to support decisions. 
Some may be uncomfortable working with data, and may rely on other professionals to 
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interpret them. Many principals report they need help in interpreting test results 
(Herman, Winters, & Golan, 1990). 

Needs of administrators and policymakers at local schools and districts have not 
received much attention in the development of systems of education indicators. This 
sourcebook is targeted at school and district staff, including principals, teachers, 
superintendents, and other management staff. Our aims are simple: to demonstrate that 
indicator data can be used as tools to improve school professionals’ ability to make 
decisions and monitor progress over time, and to help them design processes for doing so. 

The major goals of the sourcebook are the following: 

• To describe ways in which districts and schools can develop and use 
indicator data; 

• To show ways in which indicators and indicator systems are useful as 
tools; 

• To propose a process to move from policy questions to indicators, to 
collecting data, to use of information for decisionmaking; 

• To provide a set of resources and examples for school and district staff; 
and 

• To explore how districts and schools can make use of and benefit from 
information derived from national and state indicator systems. 

With the information, process model, and resources presented in this sourcebook, 
school and district-based staff and administrators should be able to design, initiate, and 
implement a local indicator system, including establishing standards and monitoring 
changes over time. Using such a system will provide local systems with hard data to help 
in decisionmaking. 

Through its work, the Metropolitan Educational Trends and Research Outcomes 
(METRO) Center at Southwest Regional Laboratory (SWRL) seeks to improve the 
quality of information that is available at school and district sites for teachers, parents and 
administrators, and to promote the systematic use of data by principals and other 
administrators. Many metropolitan schools in this region are adopting formal programs 
for site-based management. However, little attention is being given to the kinds of 
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information and information services that site-level managers will need to do the job 
expected of them. Our objectives are to increase the capacity of site-based managers to 
make data-based decisions that lead to improvements in the quality of instruction. 

What Is a Quality Indicator? 

Statistics that measure important aspects of a system or provide information about 
the condition or health of a system are called indicators. One superintendents’ group 
views indicators as “vital signs” regarding the health of the educational program and that 
point the way toward its improvement (Massachusetts Association of Superintendents, 
1991). Indicators are generally useful in a policy context to assess how a system is 
working and whether progress is being made. Indicator systems refer to models of the 
central components of the entire educational system along with indicators for measuring 
each component (Shavelson et al., 1987). They provide a means for determining the 
types of changes that might be made to improve schools or districts. Indicators are 
becoming increasingly common in local school districts, where they are beginning to be 
used as management tools. However, with the exception of scores on standardized 
achievement tests and national dropout rates, education does not have a commonly 
accepted set of indicators to gauge its quality. 

We can borrow from the field of economics, which is developed in this area. 
Everyone knows about several “leading economic indicators”: the Dow Jones average, 
the unemployment rate, and the Gross Domestic Product (GDP). These measure the 
general economic health of our nation. Although economists in Washington, D.C., argue 
endlessly about the reliability of unemployment figures (because of differing definitions 
and methods of data collection), the rate is generally accepted as a good barometer of our 
economy’s health and also is estimated at the state and local level. One problem in the 
education field, which is somewhat related to the unemployment figure arguments, is that 
states and districts use different ways to count schools and enrollments and use different 
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definitions for commonly used terms like dropout. Until 1987, states used over 10 
methods to count schools and to report enrollments (CCSSO, 1986). 

More potential indicators can be used in any given school or district, or even by 
state and federal policymakers. Consequently, there must be criteria for selecting them. 
One set of criteria maintains that indicators should: 

• Measure the central features of schooling; 

• Measure what is actually being taught or considered important to 
know; 

• Provide information that is policy-relevant; 

• Focus on the school site; 

• Allow for fair comparisons across schools; and 

• Maximize the usefulness of the data collected and minimize the 
burden of collecting it (OERI, 1988, p. 5) 

Although indicators can be useful in making decisions that will improve schools, 
there are some things that indicators cannot do: 

• Set goals and priorities. Educational goals and priorities are 
established by the public through its elected representatives. The 
information generated by an indicator system can inform those 
objectives, but it is just one factor among many in shaping decisions 
about policy preferences and priorities. 

• Evaluate programs . Social indicators cannot substitute for a well- 
designed, in-depth social program evaluation. They do not provide 
the level of rigor or detail necessary. 

• Develop a balance sheet. Social indicators lack the common referent 
available to economic indicators... education cannot put each of its constructs 
on a common dollar metric (Shavelson et al., 1987, p. 8). 

Why Do We Need Indicator Systems? 

Although economic indicators have been in use for quite some time, education 
indicators are a relatively new phenomenon. How did they originate? 

The publication of A Nation at Risk (National Commission on Excellence in 
Education, 1983) led to much policy activity, particularly at the state level, designed 
to improve schools. It has been difficult, however, for the public, policymakers, and 
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educational professionals to judge the effectiveness of reforms. Early attempts at 
tracking such effects using indicators, such as the Secretary of Education’s Wall 
Chart, received much criticism based on two arguments. First, the use of the 
Scholastic Aptitude Test (SAT) and the American College Testing Program Test 
(ACT) was criticized because of extreme variations in the makeup of the population 
of students who took these tests in different states. Second, the Wall Chart data 
provided insufficient information about policies and practices that were amenable to 
change and would facilitate increased student achievement. 

Since A Nation at Risk was published, there has been some extensive work, much of 
it funded by the National Science Foundation, to create new and better models and 
indicators of the quality of American education primarily to measure the progress of 
educational reform. Most of the work on education indicators has dealt directly with 
problems of monitoring changes in teaching science and mathematics, especially in junior 
and senior high schools. However, the questions that these indicators seem best suited to 
answer are fairly free of science or mathematics subject matter. Therefore, most of the 
work on indicator systems appears to be quite generalizable to many areas of the school 
curriculum. 

In addition to much federal activity in the area of indicators, by 1991, over 35 states 
had developed indicator-based accountability systems, and many more are in the process 
of doing so (CCSSO, 1991). Current systems often focus on student achievement, and 
education accountability is generally defined in terms of accountability for outcomes 
(Malen & Fuhrman, 1991). Several states (including Arizona, California, and Nevada) 
have passed laws mandating school or district-level report cards. Although 
accountability per se is not a new phenomenon, the focus has changed in the past few 
years from a top-down state-level external control model to a focus on local cooperation 
with the state department of education and a local emphasis on self-improvement over 
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time. However, there is a lot of variation among states in the degree to which the state 
government has strict control over districts. 

The public demand for accountability has forced school systems to produce 
information on student outcomes, thus the major focus of most accountability systems 
and report cards is on achievement outcomes. Recently, the focus has shifted from 
almost exclusive use of traditional multiple choice standardized tests to greater use of 
performance-based tasks tied to levels or standards of performance. 

Why do we need indicator data? The generally accepted purposes of indicators are: 

* to measure the health of the system; 

♦ to monitor the progress of the schools over time; 

♦ to provide descriptive data about the system, including strengths 
and weaknesses; 

• to provide clues about how components of a system are related; 

• to provide accountability systems with hard data and diagnostic 
tools; and 

• to aid in federal, state, and local decisionmaking. 

Shavelson et al. (1987) identify five functions that an indicator system might serve 
in the policy context of a national system of indicators. Four of them are equally relevant 
to management and policymaking at school sites: 

• describing status, such as level of participation in science or 
mathematics by ethnicity, gender, and social class; 

• providing an early warning by identifying emerging trends and 
problems such as sharp declines in achievement by certain 
subpopulations; 

• identifying policies that appear to be succeeding or unintended 
consequences of policies that have been put into effect; 

• supporting leadership in school reform, such as information that 
shows to what extent students are engaged in "hands-on" science. 

In practical terms, what exactly can indicators and indicator systems do for school 
and district staff? According to one large district in Maryland that has developed such a 
system, indicator systems are invaluable tools for teachers and administrators 
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(Montgomery County Public Schools, 1991). After instituting a School-based 
Instructional Monitoring System (SIMS) in 23 pilot schools in 1991, and training 
principals and teachers in its use, Montgomery County ninth graders hit an all-tune high 
in the percentage passing the Maryland Functional Reading and Math tests. Minority 
students made large gains as well. Three high school principals with high pass rates 
credited the SIMS system for helping them boost student scores, mainly by making it 
easy to quickly pinpoint students who needed help. According to an assistant 
superintendent in charge of the program, SIMS “gives principals and school leadership 
teams the mechanisms to monitor and evaluate their own effectiveness, as opposed to 
waiting to read a newspaper article to tell them how they’re doing with kids.” (Nurmi, 
Feb. 19, 1992, p. 1). How was this system useful? It helped teachers and principals to 
quickly pinpoint students who need help; plan instruction effectively; monitor school and 
student progress over time; monitor progress of special needs students; gain access to 
school and student information; and gain local control over student data (MCPS, 1991). 

Indicators within such a system are not exactly the same as information in a 
traditional student information system. Instead of being external to the building, dated, 
and aggregated as in traditional systems, indicator systems can be internally built and 
controlled, easily kept up-to-date, and individualized. School staff can get involved and 
become knowledgeable about their school. With an internal system, staff will have more 
than data or statistics; they will have information they can use to make decisions, such as 
placement in special language programs or tutoring, amount of homework needed, or 
degree of availability of academic or honors classes. 

Because indicator systems are based on models of schooling that propose 
relationships among inputs, processes, and outcomes of schooling, they can be used to 
decide whether changes in policies and programs are warranted. For example, 
disaggregating information about student enrollment in particular courses provides 
information that can help explain differential achievement Girls, for example, are less 
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likely to enroll in higher level mathematics courses than boys, and their lower 
achievement scores seem related to this fact In a high school, staff can look at local data 
that show differential course taking and achievement and decide what steps to take to 
increase female enrollment in math courses. 

In 1989, a group of school superintendents, citizens, and staff from the 
Massachusetts Department of Education began a project to develop and test the 
usefulness of an array of background, process, and outcome indicators for public schools. 
In 1990, the project became a formal program of the Massachusetts Association of School 
Superintendents. The group is working to create a grass-roots approach to accountability 
that focuses on voluntary self-improvement The association is working to get teachers 
involved in creating indicator systems by providing resources and stipends (Appendix C 
gives examples of indicators for school systems developed by the Massachusetts 
Association of Superintendents, along with a sample of data sheets used to collect 
indicator information on a spreadsheet). 

A recent report of this group details the project and presents a case for the 
development of school indicators at the local level. It asserts that benefits to 
superintendents include the following: 

• Improve the effectiveness of the school system and provide 
diagnostic tools for discovering improvement opportunities and 
allocating resources; 

• Increase job satisfaction for school system employees by providing 
“how-are-we-doing” feedback; and 

• Generate community understanding and support for the school 
system. Progress reports can help the community develop pride in 
the school system’s accomplishments (Massachusetts Association of 
School Superintendents, 1991). 

The Massachusetts group and others maintain that data or indicators have generally 
had little direct influence on planning or policymaking (David, 1988). There are several 
reasons for this. Indicators are only one of many sources of information. Many school 
decisions that can influence policy are made informally, and even formal decisions can be 
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made with little reference to a data system or indicator system. Because of this, data are 
not often collected, analyzed, or presented around particular issues. However, an 
indicator system built on a model of schooling provides a framework for developing 
indicators local staff will use. According to the Massachusetts group, an indicator system 
can be used for decisionmaking through strategic planning using site-based management 
(Massachusetts Association of School Administrators, 1991). There is some evidence 
that if school and district staff are involved in the creation of a districtwide indicator 
system, they are more likely to provide accurate information to make policy changes 
based on the data (McLaughlin & Pfeifer, 1986). 

Using the example of access to high quality curricula, local staff could decide to 
improve the quality of the curriculum in certain classes, such as eighth-grade algebra. 

This would involve allocating resources, in terms of staff time and funds for staff 
development, differently from current practice. In addition, staff could decide to provide 
additional counseling to students to increase the numbers of certain subgroups that select 
challenging courses. 

School sites have a stake in knowing more about dependable associations that exist 
between conditions of schooling and desired outcomes. Local managers are involved in 
making decisions having impacts that are intended to be long term. In one way or 
another, these decisions are attempts to deal with resources and how they are to be used; 
delivery of services to diverse subpopulations of students; standards for what will be 
taught, who will teach, and what students will be eligible to participate; goals in 
achievement that will be pursued; and outcomes that will be accepted. 

As mandatory report cards or periodic quality reviews of districts and high schools 
become increasingly common, it is likely that indicator systems will be used more in 
districts, as they can provide evidence of school improvement. For example, in 
California, every district and school is bound by the state Department of Education’s 
Program Quality Review that mandates reviews every several years. It builds on a school 



improvement model involving content standards. Quality criteria include specific aspects 
of the curriculum in the content areas and schoolwide policies and processes that shape 
and support instruction. In addition, school performance reports which report on how a 
program compares with quality criteria are released every year. Indicator systems will 
facilitate the implementation of such reviews and improvement efforts, whether or nor 
they are mandated. 

Despite accountability pressures and their obvious benefits, school use of indicators 
is not widespread for several reasons. Few school or district staff have participated in the 
creation of indicator systems, thus may not feel ownership. Some local schools and 
districts will participate in collecting data for national or state indicator systems, but the 
data they generate is not likely to be information they can use. Another reason may be 
that most indicators have focused on student achievement as measured by traditional 
standardized tests, which some believe lack validity. Many principals believe that test 
scores receive too much attention because of emphasis placed on them by state and 
district officials. Another reason may be that state reports are often too complex and 
contain too many indicators to be useful to local school staff. 

There are other local concerns about the use of indicators. Cost is part of the 
problem. In particular, indicators that require fairly precise accountings of teacher and 
student engagement or specific activities in the classroom (process indicators) over an 
extended period may be put aside. However, some data available on the school level (for 
example, course offerings or teachers assigned out of field) may be easily obtainable for 
little or no cost in schools. Additional concerns about the use of indicators include unfair 
comparisons with others, teaching to the tests, and the belief that some indicators are 
impossible to measure precisely. 

Many of these concerns can be mitigated if local schools design their own indicator 
systems. Some superintendents believe that national and state indicators are not sensitive 
to the central issues of schooling, thus the local schools should design their own 
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indicators (Massachusetts Association of School Superintendents, 1991). Local districts 
and schools do have needs for information that will support local policymaking, and they 
have a growing capacity to satisfy more of their own needs. Many districts have very 
good technology and easy-to-use software for data management. The investment needed 
to provide training and staff development that will greatly enhance the capacity of sites to 
generate and main tain databases of their own is modest. And the payoffs in terms of 
community support, school and district staff involvement and job satisfaction, and the 
availability of diagnostic tools forjudging progress over time should make indicator use 
desirable to local staff. 

The purpose of this sourcebook is to help local staff develop and use indicators so 
the decisions they make are more likely to lead to the outcomes they desire. It is 
organized in the following way. 

Section II discusses characteristics of some indicator systems, the major indicator 
realms, and examples of existing indicator system models. Definitions of content, student 
performance standards, and school and system delivery standards are provided. A 
general process to transform indicator systems into specific indicators and then into 
statistics and information for decisionmaking also is presented. 

Section III is practical and hands-on. A process model for local decisionmaking is 
proposed that moves from policy questions to indicators to collecting data to use of 
information for decisionmaking. Criteria by which to choose indicators are presented. 
Ways in which staff can use indicators for decisionmaking are described. Examples of 
models and indicators used commonly are presented. Probable future trends also are 
discussed, including federal, state, and local student information systems developments. 

Section IV is a resource section that presents information and material that may be 
useful for school and district staff in developing indicators, report cards, or assessment 
systems of their own. Resources that were selected were those that may provide useful 
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information, guidance, and assistance to local administrators. The appendices include the 
following: 

♦A glossary of indicator and assessment terminology; 

♦Examples of state/district report cards; 

♦Examples of indicators for school systems; 

♦Key dimensions of state performance accountability systems; 

♦Examples of content and student performance standards; 

♦EXPRESS system description; 

♦California Student Information System (CSIS) data categories; 

♦Resource organizations and contact information; 

♦Criteria for evaluation of student assessment systems; 

♦CRESST: Assessments in Practice Data Base Protocol; and 

♦The National Education Goals. 

Section II. Indicator Systems 

Indicator Systems: Existing Models 

Since 1983, there has been a great deal of work in designing systems of education 
indicators. Most major projects or working groups have dealt in one way or another with 
creating some kind of national model that captures the most salient features of schooling 
and associated outcomes. As previously noted, indicator systems refer to models of the 
central components (such as staff, curriculum, and fiscal resources) of the entire 
educational system, along with indicators for measuring each component (Shavelson et 
al., 1987). No general comprehensive indicator system is available for education. 

Indicator systems need to be useful for policymaking. They need to reflect theory 
and research about the relationships among the variables indicators represent. Then, 
when a policymaker looks at outcomes of interest, he or she has prior information about 
what inputs and process may need to be changed to improve outcomes. 
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Most of the recommendations for indicator systems feature some version of an 
input-->process-->output model that flows from characteristics of the community and the 
population served, through characteristics of the school it>elf, to characteristics of learner 
outcomes. Student achievement is the primary outcome in all of the indicator models, 
although other variables, such as enrollment in advanced courses and occupational or 
career choices made after graduation are considered. 

Several examples of indicator systems may be useful to local policymakers who 
want to develop their own systems. A very simple model of schooling (Figure 1) was 
used by Raizen and Jones (1985, p. 12) to select the National Research Council's (NRC) 
set of education indicators. It includes four realms (teachers, curriculum, instruction, and 
S achievement), three input variables (teacher quality, teacher quantity, and curriculum 
content), two process indicators (time spent in courses and enrollments) and one output 
variable (student achievement). Two additional variables, expenditures and public 
attitudes were considered by the NRC committee. These variables were not included in 
the selection of indicators, mainly because the committee could find no strong research- 
based relationships between these variables and schooling outcomes. 

Figure 1 

National Research Council model. 
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Later models were more extensive and included considerably more detail. For 
example, RAND's basic model of schooling (Shavelson et al., 1987, p. vi) adds details in 
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all three components of the input— >process-->output model, as shown in Figure 2. 
Outputs from this model include more than student achievement (although it would very 
likely be the primary output), and inputs include fiscal resources and student background 
along with teacher quality. In addition, school and curriculum quality, and teaching 
quality and instructional quality are added to the processes realm. 

Figure 2 

Schooling components included in the RAND model. 
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A more complicated model of an educational system that also shows links among 
elements also was presented by Shavelson et al. (1987) (see Figure 3). It includes the 
same basic domains as does the model in Figure 2, but differs in that it uses links. The 
links propose that data in different realms can be linked analytically, although not always 
in a causal way. For example, most research demonstrates that teacher quality is related 
to school quality, and that instructional quality is related to student achievement and 
participation. 

A more recent model (National Forum on Education Statistics, 1990) reflecting a 
consensus from a broad cross section of education policymakers in state and federal 
agencies describes an education statistics system that covers four domains. 
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The four indicator domains are: 

L Student and community background statistics 
EL. Education resource statistics 
Fiscal resources 

Human and nonhuman resources 
HI. School process statistics 
Implemented curriculum 
Teaching quality 
School environment 
IV. Student outcome statistics 
Student achievement 
Student participation and progression 
Student status after high school 
Student attitude and aspirations 

This model’s domains are similar to those in the RAND model. 
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Figure 3 . Linking Elements of the Educational System 



P 




Source: Richard Shavelson etal. Indicator Systems for Monitoring Mathematics 
and Science Education, the RAND Corporation, August 1987 



Another indicator system that may be useful is the one developed by the State 
Education Assessment Center. In developing the Council of Chief State School Officers 
(CCSSO) model for states, the Center first identified a set of “ideal” indicators (Blank, 
1986), based on a synthesis of results from various models. From that set, a list of 
“priority” indicators were selected based on three criteria: (a) importance and utility of 
an indicator, (b) technical quality of data that can be obtained; and (c) feasibility of 
obtaining required data (p. 7). The CCSSO project (Blank & Dalkilic, 1990) identified 
1 1 specific indicators along with anticipated data sources as seen in Figure 4. 

Figure 4 

CCSSO Priority Indicators 



Student Outcomes 
Student achievement (NAEP) 

Student attitudes/intentions (NAEP) 

Instructional time/participation 

Grades 7-12 course enrollment (state data collected by CCSSO) 

Elementary minutes per week (NCES Schools & Staffing Survey: SASS) 

Curriculum Content 

Students’ opportunity to learn (data not available) 

School Conditions 

Class size (SASS or CCSSO state data) 

Number of course preparations per teacher (SASS or CCSSO state data) 
Course offerings per school (SASS or CCSSO state data) 

Teacher Quality 
Courses/credits in science/math 

Teaching assignments by certification field/subject (CCSSO state data) 
Equity 

Gender and race/ethnicity by student or teacher indicator (CCSSO state data) 



To develop a local indicator system, it is possible to start with the basic input- 
>process->output design and indicator realms from any model, and to include or exclude 
indicators of special concern. Some models use an input— >process-->output design, but 
include special details that reflect particular concerns of their designers. For example, 
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Hall et al. (1985) suggested adding educative difficulties (pupil's capabilities, motivations, 
handicaps, English language facility, out-of-school supports, etc.) as one of three 
background variables because "...pupils who enroll in some schools enter with cognitive 
accomplishments and capabilities, motivations, and out-of-school environments and 
resources, which make educative efforts easier and less complex than those in other 
schools." (p. 9). Romberg (1987), on the other hand, specifically omitted instructional 
time because it seemed like an overly simplistic way of measuring curriculum quality, 
and had no observable link to quality. 

The School Reform Assessment (SRA) project (McDonnell, et al., 1990) designed a 
model for identifying coursework indicators that would fit within a larger input- 
>process— >outcome model of the schooling process. One objective was to probe what it 
means to measure or describe course content going beyond a title or a syllabus. Another 
objective was to ascertain the feasibility of actually obtaining in-depth information about 
coursework, using relatively low-cost methods, such as teacher surveys. The model for 
student coursework included four basic elements (p. vi): 

• Topic coverage; 

• Instructiona 1 strategies; 

• Curricular objectives (e.g., emphasis on concepts and processes in 
comparison to basic skills); and 

• Teacher qualifications. 

A larger model of the schooling process was not specified as such, but indicators 
identified by the project could easily fit within several of the models that have already 
been reviewed (e.g., Raizen & Jones, 1986; Shavelson et al., 1987). These elements fall 
within several domains, including teacher quality, curriculum quality, and instructional 
quality. 

A s imil ar project involves work by a congressionally mandated Special Study Panel 
on Educational Indicators (Burstein, 1991). In contrast to other models that use the input- 
->process->outputs structure, the panel developed sets of indicator realms tied to six 
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“enduring” issues : 

• Knowledge, skills, attitudes and dispositions for well-educated 
citizens in a democratic society; 

• Quality of schools; 

• Readiness for school; 

• Societal support for education; 

• Educational contributions to economic productivity; and 

• Equity in opportunities, experiences, and results for children at risk 
of school and societal failure. 

These six realms result in some of the same indicators as in the other models of 
schooling, but some are supplemental. For example, equity is not an indicator realm in 
most other models. 

The models presented here share many characteristics. Most importantly, they 
reflect research findings about relationships among significant components of schooling. 
The models may be useful to school and district staff who want to know how variables 
are related to other variables. The model selected points to the indicators used. 
Consequently, the model selected by district staff should be one that includes domains of 
particular interest or concern at the local site. For example, if there is local concern about 
the quality of teachers, the model chosen should include sufficient indicators in that 
domain to allow defensible judgments about teacher quality and decisions about how to 
improve it. For example, indicators could include teacher major, number of classes taken 
in field, number of years of experience, and recent in-service in the teaching field. 

Schools and districts need to review various models of schooling and examples of 
indicator systems, and choose indicator realms for their model that are useful for 
decisions to be made at their level. A school or district team can identify a basic model 
such as the NRC or RAND model and decide on which indicator realms to include. In 
particular, which variables in which realms can they influence and are related to 
important outcomes? For example, if a new high school mathematics curriculum 
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focusing on problem solving is instituted, what is the impact over time on student interest 
in math, student achievement, and course taking? Or, what is the impact of increasing 
teacher planning time (teacher quality) on amount of homework assigned and degree to 
which homework is checked (teaching quality)? Local staff members can develop their 
own models or use pieces of others’ models. Feasibility, cost and burden all play a role 
in choosing indicators. It is generally recommended that fewer rather than a greater 
number of indicators be used for decisionmaking. This is generally in the ball park of 20 
or fewer. 

Although all of the indicator system models presented have some advantages, we 
recommend the simple RAND model (Figure 2), as it includes most of the important 
domains that many local schools and districts use or report These include fiscal 
indicators, teacher quality indicators, student background indicators, school quality 
indicators, and achievement indicators. 

The Role of Standards and Assessments 

Since many accountability systems today are outcome-driven, two major issues in 
developing local systems are the role of standards and assessments. The National 
Council on Education Standards and Testing (NCEST) sees education standard as a 
generic term, generally referring to the knowledge and skills students should possess. 
Webster’s Dictionary contains two general meanings for "standard." The first is 
“something established by general consent as a model or example to be followed.” The 
second is “a definite level or degree of quality that is proper and adequate for a specific 
purpose.” The National Council of Teachers of Mathematics (NCTM) defines standard 
as a statement that can be used to judge the quality of a mathematics curriculum. 

District and school administrators have always tried to answer a general question 
about student performance: How are our students doing in comparison to some standard? 
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Generally school personnel have used grade level or average achievement as a reference 
point. 

The major questions about standards have been: 

• Are students performing at or above grade level? 

• Are students performing “adequately” when compared to similar 
schools or districts? 

• Are students maintainin g or improving their performance over time? 
(Herman, Winters, & Golan, 1989) 

Few administrators generally ask, “What do these scores mean in terms of actual 
student performance?” (Herman, Winters & Golan, 1989, p. 9) 

We are seeing the beginning of a shift in regard to the type of standards we use to 
judge performance. Using a norm-referenced framework, in which each student’s (or 
school’s) performance is only judged in relation to others’ performances provides 
information with limited utility. Asking questions such as: “What percentage of students 
are at grade level?” “How many students scored above the 50th percentile?" “How did we 
do on the state test compared to similar schools?” does not provide information that can 
lead to changes in curriculum and instruction that, in turn, can have an effect on student 
achievement 

Criterion-referenced approaches differ from a norm-referenced framework in that 
student performance is assessed against a set standard or cut point (e.g., 75% correct). In 
the emerging framework, a similar criterion-referenced approach is used in which each 
student’s (or school’s) performance is assessed against clear external standards of 
performance. Schools can use such information to identify performance gaps in a 
particular domain, such as teaching quality. The indicators collected in other domains 
identified by the schooling model can then be used to determine potential improvements 
in curriculum, teacher quality, etc., that can address the gaps. . 

The American public has recently become aware that traditional norm-referenced 
scores and methods of comparison cannot tell us what students can actually do. In 
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response to public concern about the absence of national standards keyed to world-class 
levels of performance, the National Education Goals Panel (NEGP) is monitoring goals 
and objectives for the year 2000 that were agreed upon by President Bush and the 
National Governors’ Association (Education Week, March 7, 1990). Mechanisms for 
tracking the progress of the nation and the SO states in meeting these goals are being 
developed by a National Education Goals Panel that includes representatives from the 
National Governors’ Association, the Bush Administration, and majority and minority 
leaders of the House and Senate. The Goals Panel’s annual report on the attainment of 
the National Education Goals may provide school and district-based professionals with 
ideas about indicators and systems they may wish to adopt for their own purposes. The 
Goals Panel plans to publish a handbook for local administrators who want to use the 
indicators chosen by the National Education Goals Panel to produce its own report. 

In the first Goals report (NEGP, 1991), objectives (standards) and indicators for all 
six national goals wore displayed at the national and state level (see Appendix K for a list 
of the six national education goals and objectives along with indicators for Goal 3: 
Student Achievement and Citizenship and Goal 6: Safe, Disciplined and Drug-free 
Schools. In addition, an example of state-level indicators on the goals and objectives is 
presented). An example of a goal and allied objective is the following: Goal 4: “By the 
year 2000, U.S. students will be first in the world in science and mathematics 
achievement.” An allied objective (standard or target) is: “The number of teachers with 
a substantive background in math and science will increase by 50 percent.” 

In response to national concerns about our inability to measure our nation’s 
progress toward the national education goals, the National Council on Education 
Standards and Testing (NCEST) was convened by the National Education Goals Panel. 

In early 1992, the Council issued a report that tries to move the nation toward adopting 
high national education standards for all students and recommends that new forms of 
student assessment be developed to determine progress toward national standards for and 
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methods of assessing student and school progress (National Council on Education 
Standards and Testing, 1992). Standards and assessments are the two cornerstones of this 
group’s platform. 

Student performance standards are the most well-known, but they are only one type 
of standard. The Council’s Task Force on Standards recommends content and 
performance standards for students, as well as school and system delivery standards. 
Standards are more specific than goals, and specific levels are set in objectives associated 
with standards. The Task Force recommends that overall standards be set in subject 
matter areas, including English, math, science, history, and geography. In each content 
area, an overarching statement “should describe in brief and general terms a vision of the 
nature of the education standards for the content areas” (NCEST, 1992, p. E-5 ). One 
example of such a statement is the one used in the new California Mathematics 
Framework. “Mathematical power” is described as: “Mathematically powerful students 
think co mm unicate, drawing on mathematical ideas and using mathematics tools and 
techniques” (California Department of Education, 1991). 

Content standards define specific subject matter a student should know or be able to 
use. The Council’s Task Force defines content standards as standards that describe the 
knowledge, skills, and other necessary understandings that schools should teach for all 
students to attain high levels of competency in the subject area. These content standards 
become targets for creating assessments. They also become targets for teachers who are 
constructing curricula (Tucker, 1992). Curriculum frameworks used by some states (as in 
the California Mathematics Framework),are assumed to be equivalent to content 
standards. The NCTM Curriculum and Evaluation Standards for School Mathematics 
also are considered content standards (see Appendix E for examples of NCTM content 
standards). An example of a standard and two objectives from the NCTM Standards for 
the elementary grades is: 
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Standard 12: Fractio ns and Decimals 

In grades K - 4, the mathematics curriculum should include fractions and decimals 
so the students can — 

• Use models to relate fractions to decimals and find equivalent 
fractions; and 

• Apply fractions and decimals to problem situations (Working Groups 
of NCTM, 1989). 

To translate the general content standards into the specific language of the 
classroom teacher who is charged with implementing them, a group of teachers at Seeds 
University Elementary School at UCLA prepared mathematics objectives derived from 
the California state framework and classroom practice (see Appendix E). These are 
concrete objectives for the classroom broken down into early childhood, lower, middle, 
and upper grades. Examples of several objectives relating to decimals and fractions are 
the following: 

• Identify, read, and compare fractional portions of an object; 

• Add and subtract like fractions using manipulatives; 

• Find equivalent expressions for decimals and fractions; and 

• Apply numerical operations in a problem-solving situation. 

Some work in developing content standards has been done in the areas of science, 
U.S. history and geography. In science, Project 2061 established by the American 
Association for the Advancement of Science has developed conceptual reports in the 
areas of physical and information sciences and engineering, biological and health 
sciences, mathematics, social and behavioral science, and technology. The development 
of the standards is still in progress, but a draft is expected in late 1992, which will include 
standards and benchmarks. The state of California has developed a science framework 
(CDE, 1990) that is organized around the themes of science, including energy, 
evaluation, patterns of change, scale, and structure, stability and systems and interactions. 
In addition, the National Assessment of Educational Progress (NAEP) developed science 
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objectives in 1990 that included topics within three topic areas* life science, physical 
sciences, and earth and space sciences. It also includes three aspects of the nature of 
science: the nature of scientific procesess, the nature of values and principles, and the 
nature of scientific knowledge (ETS, 1989). 

In the area of history, the Advanced Placement Program sponsored by the College 
Board provides a detailed “course description” that covers the equivalent for a full-year 
college course. These course descriptions are designed primarily for teachers and 
department heads, and have sometimes been called a syllabus. According to College 
Board, they set the “content standards” for the course. 

The state of California has developed a History-Social Science Framework for 
California public schools, as well as an English-Language Arts Framework and a Foreign 
Language Framework. Currently national groups are working on the development of an 
arts framework and a geography framework. 

According to the Goals Task Force, student performance standards establish the 
degree or quality of student performance in the subject matter set out in the content 
standards (see Appendix E for examples of student performance standards). Outcome 
standards also can be broad statements of what a student should be able to do. For 
example, many districts and schools choose “to communicate effectively” and “to work 
collaboratively with others” as outcome standards. Examples of a range of professionally 
judged student performances can serve as benchmarks for assessing the level of quality of 
a student’s performance (NCEST, 1992, p. E-4). 

The College Board Advanced Placement (AP) exams are one example of setting 
levels of performance, in which a student receives a score of 1 to 5. The performance 
levels are externally developed by subject matter and testing experts. A score of 3 means 
that a student has done well enough to pass a college-level exam on the subject A score 
of 5 or “superior” is comparable to receiving an A in the subject at college. The NCEST 
Task Force recommends using at least three levels of performance standards, which 
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might be called “competent performance, “excellent performance,” and “world-class 
performance.” 

In the state of California, the California Assessment Program (CAP) uses a 6-point 
rating scale to judge answers to the writing and mathematics prompts. This ranges from a 
1: “purposes of the task were not accomplished,” to a 4: “substantially completed 
purposes of the task,” to a 6: “fully achieved the purposes of the task, while insightfully 
interpreting or extending beyond the task” (See Appendix E: CAP Performance 
Standards for Student Work for examples). 

In addition to content and performance standards, school delivery standards set out 
criteria to enable local or state educators, parents, and the public to assess the quality of a 
school’s capacity to educate their students in the subject matter set out in the content 
standards. They refer to students’ opportunity to learn, broadly conceived, and do not 
refer solely to fiscal indicators, such as per pupil expenditure. For example, are teachers 
in a school well-trained in the content area of the standards? Does the curriculum of the 
school cover the content material in depth so that all students can master it? Do all 
students have access to equal fiscal resources and high quality teachers? Does student 
performance on desired outcomes indicate that the school provides equal “opportunities 
to learn” to all students? (NCEST, 1991). This may entail examining educational 
services, (e.g., class size, methods of enrollment, testing practices, and staff assignment), 
as well as programs (e.g. course offerings, enriched curriculum) to identify any 
racial/ethnic or language group patterns in the provision of programs and services. 
Whatever type and quality of educational services the school offers to some students 
should be equitably provided to and used by all students. An example of unequal 
delivery would be if college preparatory courses were offered and theoretically were 
available to all students, but the students using these courses were found to be 
predominantly white. 



School delivery standards also refer to policy measures designed to give all students 
opportunity to leant. These policy-related measures are more subtle, and are harder to 
observe and measure. An example of how a tracking policy may result in unequal 
opportunity to learn is the following. In college-prep math courses, the focus is 
arnHftinip and generally includes more opportunities for students to interact with teachers 
and other students, and to engage in problem solving and writing. On the other hand, in 
remedial math courses, instruction tends to focus on basic skills development, social 
conformity, and individual rote work, and shows little student communication. 
Remediation of such disparate treatment can be carried out by changing district policies 
on tracking, changing course offerings (eliminating remedial classes), or changing 
procedures for placing students in courses. Variation in resources should not be used to 
justify and excuse variation in the quality of content presented or levels of student 
achievement. School standards can provide targets for determining whether a school is 
delivering the material to students (NCEST, 1992). 

System delivery standards are similar to school delivery standards in that they set 
out criteria for establishing the quality of a state or school district’s capacity and 
performance in educating students in the subject matter. School finance lawsuits at the 
state or district level have been the most common manifestation of attempts to reduce 
disparities (usually funding) among districts. Many today demand that states provide 
schools with the resources needed to meet education standards. In recent years, 
advocates for poor districts have begun to look beyond money issues to issues of 
curriculum quality and achievement measures. 

National delivery standards have already been set out in the National Education 
Goals Panel in goals 3 and 4, which also establish objectives or targets for student 
achievement for the year 2000. The Task Force on Standards recommends that all states 
and districts establish their own achievement targets (see District Performance Report 
Summary in Appendix B for examples). For using indicators at the local level, the 
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Massachusetts Association of School Superintendents maintain that benchmarks or 
standards are necessary for comparison purposes (MASS, 1991). Section IH describes in 
detail how a school or district can go about setting standards and targets. 

The California “School and District Performance Report Summary” (see Appendix 
B) is an example of how school and district performance standards and targets can be 
reported. Since 1983-84, California has reported annually to high schools to measure 
each school’s progress toward meeting accountability goals. California currently uses 15 
quality indicators and one overall performance value for each school and district. 

Indicator realms include achievement, curriculum, dropout rate, and college-bound 
indicators (California High School Performance Report Summary, 1990). In each 
indicator realm, for each indicator, an absolute criteria level or standard is set, and the 
percentage of students achieving that standard is reported. The school and district 
distribution on each indicator is measured by counting the number of students at or above 
a certain level and converting to a percent. For example, in the curriculum area, 
percentage completing four or more years of English is an indicator. In the achievement 
realm, percent reading at the “commendable” level and above is an indicator, and scoring 
3 or higher on the AP exam is another. One-year statewide growth targets for 
improvement for each indicator are set for schools and districts. Growth is the change in 
the percentage of students meeting set performance levels. Each positive percentage 
point indicates that 1% more students met performance levels than in the past. On the 
performance reports, the actual percentages on all indicators for 1990 are shown, as well 
as growth from base year (1987-88), one-year growth, and whether or not the school met 
its growth targets. 

California’s program has two goals: to support local action toward meeting goals, 
and to provide information necessary for schools and districts to set their own goals. This 
information allows decisionmakers to judge school performance on four levels: (a) how a 
school compares with itself over time, (b) how a school compares with all schools 
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statewide, (c) how a school compares with district performance, and (d) how a school 
compares with schools with similar student populations. To deal with vast differences in 
makeup of the student population, California compares schools and districts with like 
schools and districts within a “comparison band.” The comparison band refers to schools 
and districts that have s imilar populations. A composite score is calculated for each 
school and district, based on student population in terms of ethnicity, mobility, poverty 
and limited-English-proficient students. Schools are compared with similar schools— 
from those 10% lower than it on the composite up to those 10% higher on the composite. 
Although this method allows for controlling for differences in population makeup, it does 
not measure or report access to high-quality resources including academic classes, such 
as advanced placement (AP) physics, and calculus and well-prepared teachers. Thus, 
although the performance reports give good information on performance standards and 
outcomes, they fail to give information on school or system delivery standards. 

The sample district profile for the state of Arizona included in Appendix B (a 
dummy district) includes data and indicators on outcomes (e.g., standardized test results, 
percentage of students scoring below the 40th percentile, dropout rate), inputs (e.g., 
expenditures, teacher salaries, tax capacity, percentage LEP), as well as processes 
(student-teacher ratio, graduation requirements, programs and services). The Arizona 
report contains a few indicators or criteria that could be used to judge the quality of the 
district performance, These include comparing district-level variables, such as student- 
teacher ratio, number and percentage of students in bilingual programs, and number of 
bilingual staff with the state or county average or to a set standard. In addition, student 
test performance and dropout rates can be compared to ascertain if the district is 
providing equal opportunity to learn to all students. 

Table 1 displays the crosswalk between different types of standards and indicator 
realms commonly used in schooling models. An “x” is marked when an indicator from a 
particular realm is used as the basis forjudging whether or not a standard is met. For 
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example, the fiscal and other resources realm contains indicators that allow us to make 
judgments on whether school and system delivery standards are met, but does not 
contain indicat o rs that allow us to decide if student performance standards are met. 
Student performance standards are measured through indicators of achievement or 
participation. Content standards are judged using subject matter, curriculum, and 
instructional re alms . School and system delivery standards, however, relate to all of the 
indicator realms but student achievement and attitudes, in that the capacity and 
performance of the school or system in educating students incorporate many of the 
indicator realms. 



Table 1 

Crosswalk between Standards and Indicator Realms 



INDICATOR REALMS 
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In judging the quality of a school or system in meeting these standards, educators, 
policymakers, parents, and the public should all feel ownership for the standards that are 
set. Ownership of standards is essential if there is to be commitment to reaching them. 
Without such ownership, analyses that reveal how well or poorly standards are met will 
be useless. Several possible approaches to standard setting at the national level are 
suggested in the NCEST report. The first approach begins at the national level and relies 
heavily on professional input from throughout the nation. This is similar to the model 
traditionally used by the National Assessment of Educational Progress (NAEP). The 
second model begins at the local and state level and generates documents that are later 
synthesized at the national level. This is a more “grass-roots” approach. The third model 
starts with professional judgment and examples from the national level and looks to the 
state and local levels for continual feedback. This would be an interactive process and 
would involve all groups. The Task Force recommends the third model because rt offers 

the best chance to involve a broad spectrum of the public. 

At the district or school level, there can be parallel processes in standard setting. 
The interactive model at the district level involves the development of standards by a 
group made up of curriculum experts from schools, as well as academic consultants. 
Professionals in the district, as well as parents and members of the community at large, 
would provide feedback leading to modifications. It is possible for schools and districts 
to set standards independent of the national standards or goals. However, attention to 
examples at the national and state levels may be warranted, as a significant portion of 
citizens is likely to be interested in how well local systems are meeting national goals. 

The second important issue in developing indicator systems is the role of 
assessments. According to Roy Romer, the governor of Colorado, “Just creating 
standards is not enough. We also need a syllabus, instructional materials, appropriate 
teacher training and the right kind of assessment” (College Board, 1992). The National 
Council on Education Standards and Testing (NCEST, 1992) report recommends 
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development of regional and national assessment to determine progress toward national 
standards. Assessment snide to be created that embody the new standards created at the 
local, state, or national levels. The Council recommends that a new assessment system 
should consist of multiple methods of measuring progress, should be voluntary, and 
should be dynamic, not static. 

The Council’s Task Force on Assessment (1992) reports that there are five purposes 
of assessment: 

• monitoring progress toward the national education goals; 

• holding schools or students accountable for performance; 

• certifying individual achievement and accomplishments; 

• improving instruction; and 

• evaluating the effectiveness of schooling or reforms. 

States are considered to have the responsibility for creating assessments related to 
the national standards. They may use the assessments for different purposes, but they 
should enable their students to reach the content standards. 

To bring challenging standards, higher expectations, and improved performance to 
students, curriculum resources and instructional strategies should be integrated with 
assessment and standard setting. The current separation of curriculum and instruction 
from assessment is ill-advised, and should be changed. In order that curriculum and 
instruction strategies make sense to teachers, they must be involved in developing 
curriculum frameworks and identify the characteristics that distinguish resources and 
strategies likely to help student achievement on the national standards. Curriculum 
materials must be linked to the standards and assessments used (NCEST, 1992). 

In a recent development, the College Board has announced the establishment of an 
integrated program of standards, teaching, and assessments called Pacesetter. This is 
designed for educational reform at the secondary level, and consists of course content 
outlines similar to those used in AP courses, related assessments, and teacher 
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development opportunities. Offerings in mathematics, English, Spanish, science, and 
world history are planned. Each Pacesetter course will include an outline of course 
content and learning outcomes, strong teacher training and support activities keyed to the 
content outline for each case, classroom assessments that allow teachers to monitor and 
shape instruction , end-of-course assessments (including multiple choice and fiee- 
response questions), and a valid system for scoring end-of-course achievement tests on 
local, regional, or state levels (College Board, 1992). 

Indicators related to student outcomes need to include measures that involve open- 
ended work or actual performance by students, in addition to students responses to 
multiple-choice test items. Alternative forms of assessment should be a supplement to 
and, in some cases, a replacement for current ways of testing that focus on discrete skills 
and limit student responses to multiple choice. The technology for this kind of 
assessment is limited except, perhaps, in the areas of writing and mathematics. There has 
been much work done both at the state and local levels on the development of authentic 
writing assessments, and some work on the development of mathematics items and math 
portfolios and journals. Some sample items are available for use by district or schools; 
for example, the California Department of Education’s Mathematics Sampler (CDE, 
1991b). In addition, UCLA’s CRESST Center has an alternative assessment data base, 
which contains examples of alternative assessments (see Appendix J). 

Until current efforts to develop a consensus at the national level about what students 
need to know in science, social studies and other content areas bear fruit, schools and 
districts need to develop their own agreements on what performances they wish to 
measure and how they will do so. Although the process may be difficult at the start, the 
development of alternative assessments may be worthwhile over the long run, as they 
provide meaningful information about student outcomes. 
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Transforming Models into Indicators 

Several basic models of indicator systems have been presented and discussed. Yet, 
models of schooling and standards are only the platforms for generating indicators that 
will provide policymakers with a picture of what is happening and whether things are 
getting better or worse. Some reports on the design of indicators have stated explicit 
criteria for going from model to indicator (Shavelson, et al., 1987; Blank & Dalkilic, 
1990; McDonnell, et al., 1990). Other reports allude to general principles that were 

followed in identifying indicators (Raizen & Jones, 1985; Blank, P86), but do not 

■» 

identify criteria as a distinct step in the process that was followed in indicator design and 
development. 

Because there are numerous potential indicators, criteria are needed by which 
indicators should be selected. One set of such criteria has been developed 
(Shavelson et al., 1987). Indicators should: 

• Provide information that describes central features of the educational 
system, such as teachers’ work load or curriculum offerings; 

• Provide information about current or potential problems, such as 
changing demographics; 

• Describe educational conditions of particular concern to 
policymakers and amenable to change by policy decisions; 

• Measure behaviors rather than perceptions; 

• Provide analytical links among important components; 

• Generate data from measures generally accepted as valid and 
reliable; 

• Provide information that can be understood by a broad audience; and 

• Be feasible in terms of timeliness, cost, and expertise. Indicator data 
need to be produced within a time frame that is compatible with 
policymakers' decision cycles and within given cost constraints; they 
should also be collectable, analyzable, and reportable within current 
levels of expertise. 

Actual lists of indicators derived from models vary in detail and length from seven 
"key" indicators and six "supplementary" indicators in the NRC system (Mumane & 
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Raizen, 1987, pp. 2-4), shown in Table 2, to almost 40 indicators in RAND's model 
(Shavelson et al., 1987, p.37) of a "piggyback" 0 indicator system, as shown in Table 3. 
Table 2 

Indicators inNRC System 



Primary Indicators 



Supplementary Indicators 



Extent of student learning in mathematics 
ard science 

Extent of scientific and mathematical 
literacy of adults 

Enrollment data for mathematics and 

science courses taken by students 
in high school and the amount of 
time spent on the study of science 
and mathematics in elementary 
and middle/junior high school 

Nature of student activities during science 
and mathematics instruction 

Extent of teacher's knowledge in the subject 
matter that they are expected to 
teach 

Salaries paid to college graduates with 
particular subject-matter 
specialties who choose to enter 
various occupations 

Quality of the curriculum content in state 
guidelines, textbooks and 
associated materials, tests, and 
actual classroom instruction in 
science and mathematics through 
matching to exemplary curriculum 
frameworks along four 
dimensions: breadth and depth of 
treatment and scientific and 
pedagogic soundness 



Amount of time spent on science and 
mathematics homework 
Teacher preparation-college courses in 

mathematics and science, majors 
and minors, advanced degrees 
Teacher’s use of time outside the 

classroom spent on professional 
activities related to their teaching 
of mathematics and science 
Materials, facilities, and supplies available 
and used by teachers in 
mathematics and science 
instruction 

Level of federal financial support for 
science and mathematics 
education 

Commitment of resources by scientific 
bodies for the improvement of 
mathematics and science 
education in the schools 



Source: Mumane and Raizen, 1987, pp. 2-4. 



°The "piggyback" system is one that expands current data collection efforts, most of them by the National 
Assessment of Educational Progress. Other options considered in the RAND project would involve the 
National Science Foundation in collection of data independent of what is being done now by other 
agencies. 
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Table 3 

Indicators in RAND "Piggyback" Indicator System 



Resources 

Per-pupil expenditures 
Percent of personal income 
expended on 
education 

Beginning teacher salary 
Average teacher salary 
Class size/teaching load 
Computer use and 
laboratory 
facilities 
Resource adequacy 
Computers available at the 
school 

Experienced teachers' 
salaries 



School characteristics Classroom characteristics 

Course offerings 

Course-taking requirements Curriculum 
Teacher planning time Textbook and materials use 
Dropout rates Coverage of core topics 

Student enrollments 

Instruction 

Homework 

Student use of labs and 
computers 
Teaching methods 
Access to labs and 
computers 
Assessment 



Student achievement 
Mathematics 
Science 
of all students 
college-bound seniors 
prospective science/math 
majors 



Tsachg characteristics 

Descriptors 

Experience 

Comfort with subject matter 
Regency of education 
enrichment 



S . m dmi. p ^g.ip.atign 
Extracurricular activities 
Current math/science 
course-taking 



StudgnLcharacteristics 

Race/ethnicity 

Gender 

Courses taken 

Grades 

Socioeconomic status 



Student altitudes 
Interest, liking, etc. 
Social usefulness 
Career relevance 
Intended college major 
Conceptions of 

math/science 



Source: Shavelson et al., 1987, p. 37. 

Transforming Indicators into Data 

A great deal of mediating activity takes place before one of the indicators identified 
as an element of an indicator system becomes a data element or statistic, or, as is often 
the case, becomes an integrated set of several statistics. In a way, the indicators are really 
constructs that tend to evoke a "common" sense that something is being measured, and 
that more or less of it will tell us there has been a change in the quality of education. Just 
as a high unemployment rate is felt by the American public as a decline in the economy, a 
higher per-pupil expenditure is sensed as an increase in public support for schools, if not 
a direct increase in school quality. Higher dropout rates are sensed as a decrease in 
school quality or effectiveness. Although some indicators measure actual behaviors (e.g., 
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courses taken) and some arc merely counted (e.g., per pupil expenditures, amount of 
teacher experience), others are not directly observable (e.g., access to computers, dropout 
rates). Some indicators can not be observed direcdy, but must be inferred or derived 
indirectly through administrative records or attitude surveys. Although indirect indicators 
are not as highly regarded as others, many believe they have sufficient validity to be 
useful. 

To generate data, most all of the "indicators" must be transformed into specific 
questions or "pointers" that can be answered or quantified as part of an instrument or 
some other source for data collection. Sometimes, the source is a file from a district or 
county office where certain kinds of databases, such as certificates held by teachers or 
textbooks ordered for instruction are maintained from year to year. In generating data for 
an indicator, numbers can be taken from the file and recorded somewhere else according 
to a protocol that has been worked out ahead of time, usually with a fair amount of 
precision. For example, if certification in the subject being taught is an indicator of 
teacher quality, district personnel files can be used to determine and record the number of 
teachers teaching in their field. 

More often, the data collection involves a standardized survey, a form, a log, or a 
test that is completed by students, teachers, administrators, or clerks within an individual 
school or by an external observer who goes to a school site. For example, if a school 
selects “amount of homework assigned” as an indicator of instructional quality, it will be 
necessary for the principal and all teachers, students, and parents to agree on exactly what 
this means. For instance, the metric could be number of minutes of reading and 
mathematics homework assigned per week. A standardized reporting form would need to 
be filled out by teachers, collected at the school level, and tabulated. 

Few indicators are transformed into data in isolation from other indicators. More 
often, the questions that define a particular indicator are combined in a survey or test with 
questions that define other indicators. For example, reading and math scores are often 
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reported together. In addition, data for several different indicators are often gathered at 
the same time. For example, the same form used for reporting minutes of homework 
assigned could be used to report topic areas covered in class that week. 

There are some indicators that are difficult to transform into data. Process 
indicators, such as topic coverage or teaching methods, are constrained by large holes in 
what we are able to observe easily and cheaply. If school personnel are to analyze and 
act on indicators of opportunity to learn, they need something to go beyond titles of high 
school courses or broad topic areas. One approach is to analyze coursework by time 
spent on different topics (McDoneU, et al., 1990). However, the effort needed to include 
this kind of analysis as a meaningful part of an ongoing program of data collection is 
daunting. 

Further, it is not sufficient to look at textbooks, under the assumption that teachers 
teach the content that textbooks present. That assumption is far less likely to be valid 
now than it was in the seventies and early eighties. Even if it were valid, indicators that 
tell us how well teachers follow textbooks are not consistent with current standards for 
how mathematics should be taught and learned, since many texts in use are not aligned 
with national or state curriculum frameworks. In addition, new curriculum frameworks 
generally recommend integration of different subject matters, which is not how most 
textbooks are organized. For example, mathematics standards maintain that different 
math topics, such as algebra and geometry, can be taught together using unifying ideas or 
themes. Unifying ideas are major mathematical themes relevant in different content 
areas. They reveal general principles at work in different areas and show how the topics 
are related. The concept of proportional relationships is an example of a unifying idea. 
Seeing the common principle operating in different subjects is an important part of 
mathematical understanding (CDE, 1991c). 

Within schools and districts, teachers can develop their own indicators of 
opportunity to learn. Perhaps use of a common format for lesson plans could supply 
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needed data. Involving teachers in developing appropriate data sources has two added 
benefits: It directs attention to important issues and helps develop a commitment to 
providing this kind of information. 

Transforming Data into Information, or Decisionmaking 

For data to have power, they must be packaged as information. As the National 
Forum Guide noted: “Good data help make good policies” (National Forum on 
Education Statistics, 1990). The transformation of data into information requires some 
thought about the audience. The same data can be packaged in different ways to satisfy 
the needs and interests of different audiences and the ways in which data are likely to be 
used. The importance of considering indicator development in the context of 
information and audiences was a primary concern expressed by Mumane & Raizen 
(1988) and Shavelson et al. (1987). In both cases, indicators that were selected were ones 
that fit some assumptions about policymakers as users of information. Special 
considerations were given, for example, to inclusion of indicators on race, sex, and 
ethnicity in national models, so that sampling designs for data collection could allow for 
disaggregation of outcomes, such as student achievement data by characteristics of 
students. These features in the model were thought to be particularly important to 
concerns about equity among policymakers at federal levels. These equity concerns arc 
important at the state and local levels, too. 

Usefulness of national indicators for policymakers at state and local levels will very 
likely be limited. Where states arc most likely to see information about themselves is in 
published reports that show state-by-state comparisons. With the exception of the 1990 
state-by-state NAEP in math and the CCSSO state indicator report, profiles of indicator 
data for individual states does not seem to be a service that federal projects will provide. 
However, states may be able to generate their own profiles from databases that are 
created and distributed by national indicator projects (e.g., state-by-state NAEP). 
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Some states do provide schools and districts with “report cards” that report in 
simple format how the unit is doing on state-level indicators. Some local administrators 
have complained, however, that state reports contain too many indicators to be useful 
,and indicators chosen by the state do not always fit the needs of their district or school. 
District-designed indicator systems and indicators can be tailored to the special needs or 
conditions of the unit and could be timed to coordinate with decisionmaking cycles. It 
would be easier for local staff than for state staff to transform the district data available 
into information that could be used for decisionmaking. 

A recent report on guidelines for score reporting (Aschbacher & Herman, 1991) 
examined current practice in state reporting of assessment results and provided guidelines 
for effective reporting of information. State testing directors were surveyed and various 
assessment reports were reviewed. In 1989, 47 states had statewide assessment 
programs and 30 of these used standardized norm-referenced tests. Some states also use 
criterion-referenced tests. Over three fourths of the states provide reports to districts and 
schools. Some states prepare class or individual student reports, or special reports for 
state legislatures. Three fourths of the states prepare at least five types of reports on test 
results. The report gives some guidelines for effective reporting of assessment results. 
They also can be useful for schools and districts in terms of deriving information and 
reporting indicator data other than test data. The guidelines are the following: 

1. Know the audience and the purpose. 

2. Keep it simple. 

3. Be clear, accurate, comprehensive, and balanced 

4. Use techniques to direct the reader’s attention 

5. Suit format to purpose 

See Aschbacher and Herman (1991) for a detailed description of these guidelines, as 
well as for guidelines for creating effective graphs and tables. Herman et al. (1990) 
found that principals’ favorite format of presentation is a graph. Schools principals 
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wanted to see analyses showing relationships between test results and instructional 
programs and more data about individual students to help with instructional diagnosis. 

Section III. Developing Local Indicator-based Accountability Systems 
Process Model for Decisionmaking Using Indicators 

Indicator systems are not created overnight. According to David (1988), certain 
conditions need to be present in a school or district for the creation of an indicator 
system: 

• A climate that supports planning and the use of data; 

• A commitment to improvement by district leaders; 

• Involvement of stakeholders in the design of the data system (e.g., 
teachers and principals); 

• Technical expertise and data system support; 

• Necessary resources; and 

• An action plan. 

Communities, districts, and schools vary widely in the degree to which they 
have developed accountability systems, data systems, or indicators. There is a 
great deal of variation in the degree to which districts and schools have efficient 
student information systems and a climate that supports the use of data-based 
decisio nmakin g. Some schools with little computer support have little or no 
capacity to generate their own indicator data. Although most districts of at least 
medium size have student information systems with some capacity, many do not 
use data derived in such systems for decisionmaking. 

When certain conditions are present, the following steps may be useful in 
selecting, implementing and reporting indicator data (Blank, 1992). This model 
was developed for use by states, but can be used by schools and district staff. 
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Steps in Developing an Indicator System 

A. Selecting Indicators 

1. Develop a conceptual framework based on research and policy 
interests. The schooling model selected can serve as the 
framework. 

2. Obtain commitment and cooperation of leaders. 

3. Involve policymakers, educators, researchers, and data managers 
in selecting priority indicators. 

4. Select a limited number of indicators and hold down complexity 
in reporting. 

B. Organizing a Cooperative Data System 

5. Decide method of collecting data. 

6. Work with data users and providers to establish standards for 
producing comparable data. 

C. Reporting Comparative Data on Indicators 

7. Design data forms and crosswalk procedures. 

8. Report indicators. 

SWRL recommends a district-level strategy to implement the steps in developing an 
indicator system to aid in decisionmaking. The following are general steps for initiating 
and implementing a broad district- or schoolwide strategy. The steps in the process are 
best used together, but are presented step-by-step so that educators can choose those that 
are relevant to their own unit. Figure 5 displays the model and steps to follow in a 
districtwide strategy to develop an indicator system. The strategy has three phases: 
initiation, initial implementation, and completed implementation. 

Phase L Initiation of Districtwide Strategy 
A. District-level initiation 

1. District-level goals and objectives stated 

2. District commitment and resources 

3. Needs assessment 
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4. Accountability external to district 

B. School-level initiation 

1 . School goals and objectives stated 

2. School commitment and resources 

3. Needs assessment 

C. Official decision to start implementation 

In some instances, schools may initiate or suggest a strategy to the district and ask 
for help. In other cases, a district may initiate a districtwide strategy and ask for school 
involvement. Although it is not necessary that both schools and the district work together, 
it is preferable. As seen in successful school desegregation and school improvement 
efforts, when central offices and school boards are both involved in an improvement 
effort, chances for successful implementation improve (Anderson, 1985, p. 16). 

Thus, to optimize chances of implementing the strategy, the participation of both the 
district and schools is desirable. 

In the first phase, Initiation, district-level goals and objectives are stated by district 
staff (district superintendent, assistant superintendent, school principals, and board 
members), a commitment is made to the process by district leadership, and necessary 
resources provided. A needs assessment of some sort is necessary here. A needs 
assessment will reveal how well the district is performing in terms of good school 
practice. The initial self assessment is necessary to show the strengths and weaknesses of 
the district, as well as to identify data needs. One resource to assist districts in this 
assessment is the School Effectiveness package developed by Far West Laboratory 
(Mills, 1990). This package was designed for use by district administrators and 
principals to assess their student learning, instructional, administrative and external 
support/resource needs, and to set comprehensive goals for school improvement 
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Another possible resource is the “Program Quality Review” model used in California. 
Both are built on schooling models similar to those presented earlier in section 13. 

As a result of the initial needs assessment, the district leadership should be able to 
set goals and identify high priority issues or policy questions for the cross-role team to 
address. For example, a district may show a weakness in the area of curriculum and 
instructional quality. In addition, achievement test scores are low. Since schooling 
models generally show that curriculum quality and instructional quality are related to 
student achievement and participation, the district may want to focus its indicator system 
primarily on the process domain (teaching quality, curriculum quality) and on associated 
achievement outcomes. 

Along with a commitment, resources need to be provided by the district These 
may include allocation of funds for teacher release time, training of staff in data systems 
or use of indicators, and provision of technical assistance to schools. 

At the school level, goals and objectives are stated by the principal and teachers. 
This involves developing a consensus about the purpose of the school. A commitment to 
carrying out the process should be made by the principal and staff. Generally, some sort 
of group process in which individual teachers share their goals with their colleagues is 
used to develop consensus. A number of techniques for this exist. After goals are set, a 
needs assessment of some sort similar to the one conducted at the district level should be 
done by the principal and selected staff. Many of the “effective schools” programs 
provide a framework for the needs assessment, as previously noted. 
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Figure 5 

Steps in district-wide strategy to develop indicator system . 
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Once the district- and school-level commitment to the strategy have been made and 
needs assessments conducted, a formal decision must be made to move from initiation to 
implementation. It will probably be made by the superintendent at the district level. 
Phase n. Initial Implementation of Districtwide Strategy 
A. District-level orchestration 

1. Establish cross-role team 

2. Identify indicator domains, policy questions 

3. Select ideal indicators 

4. Ascertain availability of data 

5 . Establish small number of high-priority indicators 

6. Set standards and growth targets 

a. set district performance standards, growth targets 

b. set school-delivery standards, 

c. set district-delivery standards 

7. Assess technical capacity: Adapt or develop student information 
system to collect/manage data 

8. Establish standards for producing comparable data 

a. agree on standards and definitions 

b. design data forms 

c. crosswalk with state categories 

9. Establish district-school data links and processes 

10. Adapt or develop performance measures (if applicable) 

11. Train staff in data elements, terminology and usage 

12. Provide technical assistance to schools 

13. Communicate with all involved groups 
B. School-level orchestration 

1 . Staff involvement: teacher liaison 
2.. Staff development 

3. Establish supplemental indicators and standards 

4. Provide data to district 
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In the initial implementation phase, tangible district and school-level activities 
establish that the strategy is being put into place. Although the district office orchestrates 
most of the activities, cooperating with schools on activities is optimal. District-level 
orchestration includes a series of technical and administrative tasks. These include first 
establishing a cross-role district team. It is important to involve all stakeholders. Thus, 
the cross-role team should be made up of the superintendent or assistant superintendent, 
principals, teachers, and parents, as well as district data processing managers. 

Involvement of teachers and principals can help to increase the likelihood that indicators 
chosen will reflect practices that teachers or principals can control (David, 1988). 

Next, based on results of the needs assessment, the team identifies indicator 
domains or categories and policy questions that need to be answered and selects ideal 
indicators. In selecting indicators, teams can make use of examples of indicator system 
and domains presented in this sourcebook. For example, a district could decide to use the 
RAND model (Figure 3) with its 10 domains. “Ideal” indicators are identified. These 
could include, for example, in the area of school quality class size, number of course 
preparations per teacher and number of course offerings. Next, a subgroup can ascertain 
availability of these data either at the district or school level. Are there data collected at 
the school level on the number of course offerings or can the data be gathered? Class size 
may need to be calculated using number of teachers and number of students per school. 
Some of this information may already be collected in the schools or at the district level. 
Based on this report, the team will decide on a small number of high priority indicators. 
To start with, fewer than 20 is desirable. 

The group must then set (a) student performance standards and growth targets; (b) 
district delivery standards (and growth targets if desired); and (c) school delivery 
standards (if relevant) and growth targets (if desired). 

There are many ways to establish benchmarks or performance standards. Several 
approaches are suggested by a group of Massachusetts superintendents: 
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• Compare with one’s own history; 

• Compare with comparable entities; 

• Compare with national or statewide averages; 

• Compare with “good practice” recommendations by associations; 

• Derive analytically through use of models; or 

• Make judgments using a group process or intuition (Massachusetts 
Association ox School Superintendents, 1991). 

Some of these benchmarks are more norm-referenced (e.g., compae with 
comparable entities, compare with national or state averages), while others are more 
criterion-referenced and are more in line with new conceptions of standards (e.g., 
compare with good practice used by associations such as NCTM Standards or derive 
analytically through use of models). In addition, most schools and districts have a desire 
to track their indicators over time. 

District performance standards can be as simple as “percentage of high school 
students completing four or more years of English” and a one-year target could be set at 
2 %. In general, over 85 or 90 percent of students complete four years of English; thus, 
the target is set fairly low. In terms of achievement, a district could set a standard in 
terms of percentage of students scoring “commendable and above” and a one year target 
could be set at 4 % as is done in the California Assessment Program (see Appendix B for 
an example). Or it could set a standard or percentage of students scoring at or above the 
50th percentile (if only norm-referenced tests are given). Growth targets should be set 
after examining past trends in district-level achievement. What is a reasonable target, 
based on the district's past history? School performance standards and targets can be set 
the same way as district standards, taking into consideration specific school 
characteristics. 

Most educators today believe that there should only be one standard, instead of the 
current situation in which disadvantaged children are held to a lower “basic skills” 
standard, and in which more is expected of advantaged children. Although the same 



standard in terms of achievement could be used for all schools in a district (e.g., 
percentage scoring at or above the 50th percentile), growth targets for various schools 
could be different For example, if one particular school has a high percentage of limited- 
English-proficient (LEP) students and a high influx of immigrants, the growth target 
would be set lower than the target for a school that has very few LEP students and a low 
transiency rate. 

In terms of setting school delivery standards, a team needs to set out indicators to 
enable local educators and parents to assess the quality of the school’s performance in 
educating students in the subject matter and the progress the district is making in 
delivering quality programs and services to all students. Does the school (or district) 
“deliver” to all students an equal opportunity to learn? Here, input conditions such as 
fiscal resources and teacher quality are important, as well as process indicators such as 
curriculum quality. An example of setting criteria to assess a school’s performance is the 
following. 

A district may be concerned that its “high-quality” (qualified in the field) teachers 
are not equally distributed to various subgroups of student. Do LEP and disadvantaged 
students have equal access to qualified teachers? (e.g., those teachers who majored in the 
subject field, those who have taken recent in-service training in their field). A district 
could decide to set an overall criterion that at least 50% of its teachers majored in the 
field they are teaching in. In addition, this criterion could be examined by student 
subgroups (ethnicity, language background, SES) to see if there is equal access to quality 
teachers. 

On the outcome side, a district may ask, “Does each school within the district 
provide equal opportunity to all students to take challenging courses such as AP physics, 
AP calculus, or AP English?” “Are there differences in the degree to which students have 
access to “gatekeeper” courses such as algebra or geometry, which have been shown to 
be highly related to test achievement?” Other delivery indicators could be level of 
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disciplinary actions, differential tracking policies, criteria for assignment of teachers to 
classes, and attendance statistics. 

After setting standards and growth targets, districts need to assess their technical 
capacity to develop an indicator system, similar to how they conducted a needs 
assessment in the initiation phase. Some of the questions that may need to be answered at 
the district level include the following technical questions: 

1. How much information needed is already being collected by data 
processing in the schools and district? 

2. How will data from different sources be integrated? 

3. To what degree is it anticipated that our district will need to establish 
new channels of information that flow into and from the central 
office? 

4. How does enrollment and attendance information make its way from 
individual school sites to the district office (pencil and paper, 
electronic transmission)? 

5. To what degree do principals need help in getting the data they need 
and in preparing reports? 

6. To what degree do teachers need help in interpreting and using data on 
students? 

In the next two steps, the data processing manager will most likely take the lead. 
First, it will be necessary to adapt the district’s student information system to collect or 
manage the planned data collection. In the few instances in which districts do not have a 
student information system, a basic database program, such as EXCEL, can be used to 
develop data sheets to record indicator information, either on hard copy forms or onto a 
floppy disk program developed by the data processing staff (see Appendix J for an 
example of data sheets used in a Massachusetts district). 

Members of the team, including representatives from various schools, must 
establish standards for producing comparable data. This involves agreeing on data 
standards and definitions; designing data forms to collect data; and conducting a 
crosswalk (or match) with state categories (if they exist) to ensure comparability. Data 
element standard definitions can be obtained from the Council of Chief State School 
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Officers in Washington, D.C., or from California’s Student Information System (CSIS) 
in Sacramento, (see Appendices F and G.) 

Data forms to record indicator information can be easily developed using software, 
such as DBASE or EXCEL, which use templates. Each indicator should have an entry 
n umb er for use over time. A standardized format should be maintained by all reporting 
schools to facilitate compilation of data. A data manager should write a brief 
instructional handbook to accompany the data forms or floppy disk to ensure ease of use. 
And standardization of data entries. The instructions should be revised following review 
and feedback from other team members. 

The district team next needs to establish district and school processes to collect 
data. This may involve development of data collection hard copy forms or a floppy disk 
for data entry. The district will need to ascertain the types of training needed by the staff. 
Teachers or teacher aides may need to be trained in data element terminology and usage 
and data clerks may need to be trained in data entry. School staff may need assistance in 
working with computers or databases. Finally, the district is responsible for 
co mmuni cating with community members, staff and parents about the process, either 
through staff meetings, newsletters, or bulletins. 

If performance assessment measures are desired, adaptation of current measure or 
development of new measures will be necessary. There are performance tasks available 
in various databases (see CRESST’s Alternative Assessments database in Appendix J). 
As previously noted, the content and performance standards should be the target for 
creating the assessment. If new measures are desired, district resource teachers need to 
work with assessment experts and classroom teachers to design the assessment to be used 
to show progress in meeting content and performance standards. Use of items or tasks 
that have already been tried out somewhere is recommended, as development of 
performance assessments is a time-consuming and technical process. 
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School-level orchestration involves planning, organizing, and managing the school 
activities. First, as many school staff as possible should be involved in the process. In 



particular, one teacher liaison to the district team should be appointed at each school. 

This teacher should be given release time or a stipend, and should organize and 
coordinate the school’s indicator system and plan for improvement with the district’s 
indicator system. He or she would organize meetings to establish supplemental school 
indicators or standards, attend all district meetings, help design staff development for 
teachers and principals, and assist the principal in providing data to the district 

Each school may decide to use the indicators chosen by the cross-role district team, 
or may decide to supplement them with additional indicators. These indicators should be 
created by teachers and the principal. Each school needs to establish which indicators it 
wants to use and set standards and criteria for improvement Last the school needs to 
provide necessary data to the district 

Phase UL Completed Implementation of Strategy 

A. Collect compile, and score data 

1. Collect data 

2. Compile data 

3. Staff training 

4. Score performance assessments (if applicable) 

B. Transform data into information, report indicator data 

1 . Transform data into information 

2. Compare performance levels in indicator realms against standards 
(e.g., background/resources, teacher quality, equity, curriculum, 
school environment outcomes) 

3. Develop information for decisionmaking 

4. Create report card or indicator guide 

5. Develop plan for improvement 

C. Continuation/Institutionalization Decision 
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In the next phase, data arc collected by district staff, compiled and scored, and any 
assessment scored. Data obtained then need to be transformed into indicators and 
information for decisionmaking. When first beginning this process of developing a 
district indicator system, it is recommended that districts start with only a few critical 
indicators. Otherwise, coming to agreement on definitions, data elements, standard 
levels, etc., may be difficult within a district’s limited time frame. In addition, data 
collection, compilation, and scoring are time consuming. 

A major part of the overall process is transforming the data collected into 
information for decisionmaking. First, the district and school need to compare 
performance levels in various indicator re alms against the standards set The cross-role 
team needs to decide which data to present in the report card or indicator guide. In 
addition to test data, schools may want to report teacher/staff information (e.g., number of 
staff, pupil/staff ratio, teacher experience, teacher turnover, salaries), student information 
(e.g., enrollment socioeconomic status (SES), class size, mobility, percentage LEP, 
percentage Chapter 1, percentage free lunch) and other areas of school effectiveness (e.g., 
dropout rates, graduation rates, opportunity to learn, percentage taking AP courses). Two 
examples of district report cards using various indicators and formats are given in 
Appendix B. 

After putting together the report card, the district staff develops a plan for 
improvement. Based on its results and its targets, what level of improvement can it 
expect in the coming years? Finally, a continuation/institutionalization decision must be 
made (e.g., to continue the process over time). This may involve additional funding or 
personnel. 

Looking to the Future 

National efforts to develop and collect indicator data may not currently be able to 
provide much indicator information to local districts or schools, but there are ways in 
which the wealth that is accumulating from this work can be distributed to local districts. 



The National Center for Education Statistics (NCES) operates an annual national survey, 
the Common Core of Data (CCD), which includes both fiscal and nonfiscal data. The 
survey collects data at the school, district, and state levels, but generally reports data only 
at the district and state levels. Since 1988, a report released annually describes the 
characteristics of the 100 largest public elementary and secondary school districts in the 
United States, including such data as enrollments, number of graduating students, number 
with Individualized Educational Plans (BEP), and pupil-teacher ratios. Recent 
improvements in the CCD database, including agreement among states on common 
definitions and methods, have enabled a more comprehensive look at the nation’s and 
states' breadth and quality of schooling. In conjunction with the Chief State School 
Officers (CCSSO), NCES and its national Forum group have focused in recent years on 
developing a comprehensive, accurate, and timely reporting system for delivery to states. 

In 1989, a task force established by the NCES began examining the feasibility of a 
nationwide electronic record transfer system. This was done in the hope that local school 
staff would benefit. It was hoped that the system would encourage state and local 
information system comparability. Participants in the task force include school district 
and state education agency staff from five states, NCES staff, and CCSSO staff. 

After agreement on data standards and definitions, a pilot transmission of data in 
study of the EXPRESS system (Exchange of Permanent Records Electronically for 
Students and Schools: see Appendix E) was conducted in five states. The task force first 
developed a set of standard data elements and definitions in American National Standards 
Institute (ANSI) format for a pilot electronic transfer of students' records from districts to 
colleges. Data elements are in five areas: 

• Demographic (name, address, race, home language); 

• Academic history (prior school, course work, grades, attendance); 

• Special programs and services (program type, placement dates, 
eligibility status); 
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• Health; and 

• Test information. 

Some of the anticipated benefits of the national record transfer system include 
promoting greater comparability and standardization of student information across local 
information systems; more timely request and receipt of student records; more timely 
availability of data for use in determining placement and support services for new 
students; greater efficiency for districts with automated student information systems,; 
reduced total cost to transfer records and availability of a multifunction network to local 
agencies, which can be used for many purposes. Although this is not strictly an indicator 
system, it includes five domains (demographic, academic history, special programs and 
services, health and test information) encompassing input— process— outcomes, and 
includes data elements or statistics agreed upon by all participants. 

In a joint NCES-Census-CCSSO project, 1990 census mapping was carried out, 
converting census blocks to the nation's 17,000 school districts. School district 
boundaries were superimposed on a census map by block and the information digitized 
and converted to Census TIGER files. State coordinators assisted in the mapping. Over 
200 tabulations covering demographic characteristics will be run and data will be 
distributed to states in late 1992, along with a CD-ROM disc, and user-friendly software. 
The CD-ROM file will include data from the 1990 Census, along with the NCES CCD ED 
number to allow merging of CCD data with Census data. Variables include a large range 
of demographic population data, fiscal characteristics, as well as education context 
variables, such as dropouts and percent age free lunch. It is likely that state level and 
district-level outcome data can also be added to the database to allow analysis of 
relationships. 

One way that local districts can benefit from national developments is by using the 
knowledge gained at the federal level by solving methodology and data collection 
problems in developing such information at the local level. Many of the problems that 
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are being addressed nationally in the design of data collections are s imil ar to problems 
that local sites face on a smaller scale. For example, much of the effort in national 
projects goes into the design of data collection so that information about student 
outcomes can be disaggregated by ethnicity and gender and by other variables that 
represent school background characteristics, school resources, and school processes. 

Local sites need to be able to see many of the same kinds of disaggregations if they are to 
be able to make good policy. In other words, they need to be able to take advantage of 
the growth in knowledge at the federal level in how to convert indicator constructs such 
as class size and teacher qualifications into surveys, forms, and assessments that, together 
with designs for data collection, can develop and maintain an indicator database over an 
extended period of time. 

Involvement of local administrators and teachers in different aspects of design and 
interpretation of data has two potential benefits. First, there is a beginning of an 
investment in the most basic units of an infrastructure — local sites — for generating 
information about education. Sites that work collaboratively with national research 
projects on development of new indicators will become models for how local school sites 
can generate quality information about context and processes. Second, the methods and 
technology that grow out of these projects may be more feasible for local sites to carry 
out than for the staff of a nationwide project. Local sites do not assess costs in the same 
way as national projects. What is a true cost of data collection for a national project may 
be subsumed under “staff development” when the data collection is “owned” by site-level 
staff. Agencies who sponsor or conduct research on new such indicators should consider 
ways to disseminate findings to local sites, even before some of the findings can be 
incorporated into data collection efforts to support a national system. 

Of particular relevance to local sites are problems such as the following: 

* level of detail of questions needed to get at different aspects of an 
indicator variable; 
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• considerations about consistency in data collection from one year to 
the next that will improve the comparability of indicator statistics 
over time. In particular, what kinds of problems need to be 
anticipated in maintaining a sample across years? 

• methods used to obtain different kinds of information that are critical 
knowledge for local sites if indicators generated are to be 
meaningfully compared to national standards; 

• agreement on common definitions of important terms such as 
dropout; and 

• integration of data from various sources. 

As schools across the country continue a process of restructuring, the locus of 
responsibility for school management and policymaking shifts more to school sites. The 
fact that districts and school sites have not in the past engaged in serious efforts to 
generate data and to create and maintain indicator databases in support of decisionmaking 
should not be taken as evidence that they will not develop this kind of capacity in the 
future. The times are changing. 



Resources 



RESOURCES 



The following resources are included for the benefit of school and district staff. 

Appendix A is a glossary of indicator and assessment terminology and is provided for those 
not familiar with such terminology. 

Appendix B provides various examples of state or district report cards, including two from 
California, one from Arizona and one from Massachusetts. These are provided to give readers an 
idea of the commonly-used indicators in (generally state-driven) indicator systems. However, it 
is important to notice that although the California school and district performance report 
summaries focus on outcomes, the Arizona district profile contains fiscal and other input 
indicators, as well as performance indicators. The Massachusetts “Fact Sheet” only reports 
results for achievement scores. School and district staff can use these examples as a springboard 
for selecting their own indicators. 

Appendix C give examples of specific indicators for school systems developed by the 
Massachusetts Association of School Superintendents (MASS, 1991). It is provided because it is 
a good example of a local effort to choose indicators for a school system based on the input- 
process-outcome model. This system includes indicators for all the realms in the basic RAND 
model but one (teacher quality) and is well thought out. Also included is a form used in a 
database in the state of Massachusetts. This may be useful to district data processing personnel 
who want to create or adopt such a student information system. 

Appendix D is a list of the key dimensions of the 50 state performance accountability 
systems, including whether the system is state, local or mixed, whether or not the state has a 
comprehensive indicator system, what types of test are used in the state, and whether the state 
publicly reports data on schools, districts and the state. It also lists whether the state reports 
comparisons, if data are reported in the context of demographic facts and whether performance 
triggers rewards or sanctions. This is provided as an informational resource for those who are 
interested in state level trends. 
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Appendix E gives examples of content and student performance standards. First, the 
NCTM Standards for the elementary grades are excerpted. Then, a set of Mathematics 
Objectives created by a group of teachers at UCLA’s Corinne Seeds University Elementary 
School is presented. The teacher committee used the NCTM Standards and the California 
Mathematics Framework to create objectives to be used by teachers in classroom planning. This 
document effectively translates the NCTM content standards into teacher language, or concrete 
objectives to be taught in various content areas. The Seeds group is currently developing a 
companion assessment package to go with the objectives, which will have actual exercises and 
performance tasks to assess student performance levels. Hopefully, this document will be useful 
to teachers, principals and other local staff in implementing national or state frameworks. 

Also included here are two examples of student performance standards drawn from A 
sampler of mathematics assessment ( CDE, 1991). The first is a generic list of standards that can 
be used with many tasks. The second is a rubric specific to a task that asks students to write a 
paper on results of a smoking survey. 

Appendix F includes an overview of a national system: Exchange of Permanent Records 
Electronically for Students and Schools (EXPRESS). It is included because there are several 
district^ from five states currently participating in the pilot study, and local districts may be 
interested in participating in such a system. 

Appendix G is a description of the California Student Information System (CSIS) and 
listing of categories of data. It is included for the benefit of readers in California that may be 
interested in the upcoming system. In addition, the categories of data are commonly-used ones 
and are the same as those used in the EXPRESS system. The state of California is cooperating 
with the federal group on common data standards. 

Appendix H lists resource organizations and contact information for the readers who may 
need further help. 

Appendix I presents Criteria for Evaluation of Student Assessment Systems created by the 
National Forum on Assessment. These have been endorsed by many federal, state and local 
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groups across the country and are provided as a service to districts or schools that are developing 
new assessments. 

Appendix J is a document produced by the CRESST Center at UCLA: Alternative 
Assessments in Practice Data Base Protocol. The form can be used by any group that has created 
an assessment task and wishes to share it with others. Readers can contact CRESST for 
information on its assessment newsletter or on access to its data base materials. 

Appendix K: The National Education Goals includes a list of all six national goals, with 
their associated objectives. In addition, for goal 3: Student Achievement and Citizenship, an 
excerpt from the Goals Panel’s first report is included that displays what we now know and what 
we still need to know. Exhibitions of specific indicators of competency in mathematics and 
student drug use are included for informational purposes. An excerpt from one state’s report 
(Colorado) is also included. This information is included for those readers who may want to 
develop goals or objectives similar to those adopted nationally. 



67 




Appendix A 



Glossary of Indicator and Assessment 

Terminology 
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Glossary of Indicator and Assessment Terminology 



Accountability Goal : Accountability for outcomes; subject to giving an account. 

Advanced Placement (AP) examination. Exam given in various content areas by College Board 
to assess the level at which a student has passed. 

Constructed-r espouse Items : Open-ended written items that require students to produce a 
solution to a question. 

Content Standards: Standards that set out the knowledge, skills, and other necessary 

understandings that schools should teach in order for student to attain competency in subject 
matter (ex. NCTM Standards, California Curriculum Frameworks). 

“High Stakes” exam: An exam that is important in a student’s life in terms of placement, 
grading or promotion. 

Indicator System: Model of the central components of the entire educational system, with 
indicators to measure each component. 

Indicator: Data or statistic that measures important aspects of a system or provide information 
about the condition of education. 

Opportunity to Learn : Equal opportunity provided to students to learn the material. This 
includes topic coverage, instructional materials, well-trained teachers and curriculum 
resources. 

Outcome: Output of the educational system (e.g. achievement, participation) 

Performance Assessment; Testing methods that require students to create an answer or product 
to demonstrate their knowledge or skills. 

Reliability : Consistency and generalizability of test data. 

Report Card: checklist of key aspects of education that rates progress in meeting standards. 

School Delivery Sandards: Criteria or metric set to enable local and state education staff, 
parents, and the public to assess the quality of a school’s capacity and performance in 
educating student in subject matter described in content standards. 

Standard Setting Process: A process by which standards are developed. The three major 

approaches are a) begin at the national level and rely on professional input; b) begin at local 
and state levels and synthesize at the national level; and start with professional judgment 
and examples from the national level and look to the state and local levels for guidance. 

Student Performance Standard: Standard that establishes the degree or quality of student 
performance in the subject matter set out in the content standards (e.g. level 1 - 5 on AP 
exams) 

System Delivery Standards: Delivery standards that set out criteria for establishing the quality of 
a school system’s capacity and establish targets for student achievement. 
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Validity: Whether or not a test measures what it is supposed to measure. 

World-class Standards: The highest level of student performance standards. To ascertain 
performance at this level, it is necessary to gather information about the quality of the best 
student work in other nations. 
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Examples of State/District Report Cards 



CALIFORNIA STATE BENCHMARKS 1991 

For Measuring the Well-Being of California's Children 
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Special Education Program* Available* 


Students Served 

m&M 


Multiple Handicapped/Severe Sensory Impairment 


3 


Deaf/ Blind 


0 


Visually Handicapped 


13 


Physically Handicapped 


14 


Multiple Handicapped 


22 


Autistic 


1 


Severe/Profound 


14 


Hearing Handicapped 


24 


Trainable Mentally Handicapped 


36 


Seriously Emotionally Handicapped 


7 


Other Health Impaired 


14 


Educable Mentally Handicapped 


67 


Emotionally Handicapped 


165 


Learning Disabled 


1,067 


Speech 


401 


TOTAL Special Education Students 


1,848 



• Note: All Arizona school districts are required under federal and state 
laws and regulations to assure that all handicapped students have 
available to them a free appropriate public education including special 
education and related services designed to meet their unique needs. If 
some services are not provided, it is possible that there are no students 
requiring that service, or the students are being served by another 
agency (i.e., adjacent district, state school, county consortium or private 
provider). 



Federal School Lunch Program 1988-89* 





Elsnu 


Junior 

ti&b. 


High 

School 


Adult 


Reduced Prices 
Breakfast 


$0.20 


$0.00 


$0.00 


N/A 


Lunch 


0.40 


0.40 


0.00 


N/A 


Paid Prices 
Breakfast 
Lunch 


0.40 

0.70 


0.70 




$1.25 



Special Milk Prices 0.00 

Food Service Workers 

Full Time 55 

Part Time 1 06 

Students Qualifying for Free/Reduced Price Lunch: 1 ,059 

% of ADM Qualifying for Free/Reduced Price Lunch: 5.70% 

Schools Not Participating: 

Center School 
Big School 

* Participation in the Federal School Lunch Program is voluntary. Districts 
or individual schools in districts may opt out of the program. 



Special Service Agencies for Special Education 



Far West Developmental Learning Center 



19956 West McDowell 
Buckeye, AZ 85326 (257-2843) 

Maricopa County Special Education 
Accommodation School District No. 512 
1 1 Codidge Street 

Williams AFB, AZ 85225-7315 (892-1102) 

Maricopa Special Services Consortium 
210 South Sixth Street, P. O. Box 518 
Buckeye. AZ 85326 (386-4471) 



Preschool Program 

- 4-day program 

- Transportation and meals provided 

- Program provided free of charge 

- Program considered part of a community school 

For mor6 information regarding the preschool program, contact the 
district office. 
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ABC UNIFIED SCHOOL DISTRICT 

Arizona Education Profile 

Arizona Depanmem of Education • C. Diane Bishop, Superintendent of Public Instruction 

The ABC Unified School District is located in Maricopa County, about 10 
miles east of Somewhere, and serves kindergarten through 12th grade 
students from ABC and the surrounding environs with twenty elementary 
and middle/junior high schools and four high schools. More information 
regarding ABC is available in •Community Profiles* from the Arizona 
Department of Commerce. 



Superintendent: 

Assistant Superintendents 



Address: 

Phone: 



Dr. Duane Smith 
Dr. Glen Smith 
Robert C. Smith 
Dr. Betty Smith 
John Smith 

3800 North 20th Street, Phoenix, A2 85000 
555-5555 



School Board Members: 



Christy Jones 
George Jones 
Susan Jones 
Don Jones 
Diane Jones 



ENROLLMENT AND STAFFING 



Students 1 983-84 


1988-89 Staff 


1983-84 


1988-89 


Enrollment 19,158 


19,400 Administrators 51 


62 


ADM* K-8 10,983 


11,732 Teacher Aides 99 


77 


ADM 9-12 7,958 


6,845 Teachers 1 .053 


1,131 


Total 18,941 


18,577 Other Staff 637 


645 




Total 


1,840 


1.915 


•Average Daily Membership (Attending) 






1988-89 


ABC 


fdaricopa Co. 


Arizona 


Students per T eacher 


16.4 


18.6 


18.4 


Students per T eacher Aide 24 1 .3 


152.6 


127.1 


Students per Staff 


9.7 


10.2 


9.7 


Average Teacher Salary $32,799 


$30,605 


$29,402 


Racial/Ethnic Composition 1988-89 








ABC. 


Mari co Da Co. 


Arizona 


White 


80% 


71% 


64% 


Black 


1% 


5% 


4% 


Hispanic 


15% 


20% 


24% 


American Indian 


1% 


2% 


7% 


Asian 


2% 


2% 


1% 



1988-89 
Gifted Students 
% of Total ADM 
LEP* Students 
% of Total ADM 
Special Education Students 
% of Total ADM 

•Limited English Proficient 

FINANCIAL INFORMATION 



ABC. 


Maricooa Co. 


Arizona 


2,423 


19,292 


26,803 


13.04% 


5.92% 


4.62% 


415 


19,463 


53.133 


2.23% 


5.98% 


9.12% 


1,848 


29.440 


54,110 


9.95% 


9.04% 


9.29% 



Expenditures 

Maintenance & Operations 

Administration 
Instruction 
Instruction Support 
Operations 

TOTAL M&O Expenditures 

Teacher Retirement Adjustment 
Adjusted M&O Expenditures 

Non M&O Expenditures 
New Capital Items 
Capital Outlays 
Debt Service 
Bond Building 
Total New Capital Items 
Miscellaneous Non M&O Items 



1983-84 1988-89 % Increass 

$1,658,297 
28,137,259 
6,486,052 
9,808,115 
46,089,723 
2,140,453 
49,362,714 

1M 



$2,142,878 


29.22% 


43,516,383 


54.66% 


10,085,093 


55.49% 


13.553,789 


38.19% 


69.298,143 


50.35% 


69,298.143 


40.39% 


1988-89 


% Increase 



$4,317,559 
1 ,368,570 
0 

5,686,129 



$3,633,658 

8,952,199 

27,444,840 

40,030,697 



-15 84% 
554.13% 

604.01% 



School Plant 


1,132,538 


902,230 




Federal/State Projects 


1,381,142 


1,722,917 




Adjacent Ways 


6,584 


13,500 




Intergovernmental Agreements 


0 


0 




Indirect Costs 


25,594 


43,462 




Unemployment Insurance 


32,267 


14,246 




Other 


1,881,821 


3,340,848 




Total Misc. Non M&O Items 


4,460,946 


6,037,203 


35.33% 


TOTAL Non M&O Expenditures 


10,147,075 


46,067,900 


354.00% 


TOTAL EXPENDITURES 


58,377,251 


115,366,043 


97.62% 


Teacher Salaries (All Funds) 


$27,275,989 $35,312,85? 


29.46% 



o 

ER AC 



* These contributions to the Sate Teacher Retirement Fund were not 
included in Maintenance and Operations expendtures for 1983-84. but are 
1988-89. 



Property Ownership 1988-89 




Land and Improvements 
Building and Improvements 
Furniture and Equipment 
Construction in Progress 



$18,774,181 

91,913,148 

18,736,297 

17,062,667 



Bonds Outstanding (June 30, 1989) $77,650,000 



Property Tax Assessment August 1988 

Tax Base* Tax Rate**. 

Primary Assessed Valuation $1,329,653,362 $4.3393 

Secondary Assessed Valuation 1,469,313,443 0.8458 

* Excluding Salt River Project Assessed Valuation 
** Per $ 1 00 Assessed Valuation 



Revenue Sources 1988-89 










Local 


Cfluntt 


State 


Federal 


M&O 


$48,453,590 


$391,787 


$17,409,290 


$< 


Capital Outlay 


3,046,146 


0 


787,401 


1 


Adjacent Ways 


63,808 


0 


3,897 


i 


Debt Service 


8,315,963 


0 


18,084 


i 


TOTAL 


59,879,507 


391,787 


18,218,672 




% of District 
Revenues 


76.29% 


0.50% 


23.21% 


0.00*? 



Arizona 

$1,007,260,925 
1,959,064,330 
1,119,161,196 

‘Definitions: 



Primary Property Tax Capacity 1989* 

ABC. M aricopa. Co. 

Capacity $70,358,066 $651,704,800 

Equalization 

Base 62,089,984 1,076,065,265 

Equalization 

Aid 2,594,845 557,387.078 



% of State 



Revenues 44.66% 4.01% 


48.677. 


. 2.667. 


Program Expenditures* 








.1223:84 


1288-82 


Special Education 






Educable Mentally Handicapped 


$302,901 


493,443 


Seriously Emotionally Handicapped 


610,977 


788,439 


Hearing Handicapped 


149,849 


187,008 


Other Health Impaired 


110,064 


93,864 


Multiple Handicapped 


147,194 


213,708 


Severe Sensory Impaired 


— 


20,649 


Physically Handicapped 


43,708 


121,803 


Learning Disabled 


3,196,477 


4,332,142 


Speech Handicapped 


577,066 


996,704 


Trainable Mentally Handicapped 


326,159 


347,761 


Visually Handicapped 


133,983 


212,166 


Total Special Education 


5,598,378 


7,807,687 


Gifted 


464,376 


860,363 


Bilingual Education 


0 


452,166 


Remedial Education 


0 


0 


Vocational Education 


1 ,087,954 


1,499,335 


Career Education 


0 


0 


TOTAL (ind. in M&O) 


7,150,708 


10,619,551 


% of Total M&O Expenditures 


15.51% 


15.32% 


State % of Total M&O Expenditures 


12.64% 


13.46% 



Property Tax Capacity is the dollar amount of revenue that a given 
district would raise if it applied the qualifying tax rate on property. 

State Equalization Base is determined using the state funding formula. 
State Equalization Aid is the actual amount of state aid that each district 
is given . This figure is determined by subtracting the Actual Tax Collections 
from the State Equalization Base. Maximum usage of the available tax 
base is required to receive State Equalization Aid. 

STUDENT PERFORMANCE INDICATORS 1988-39 
Standardized Teat Results April 1989* 

U.S. 





ABC 




Arizona 


Average 


ade 


R L M 


R L M 


R L M 


R L M 


1 






43 58 49 


54 59 52 


2 


73 79 75 


53 61 60 


49 57 56 


53 55 52 


3 


65 68 59 


49 54 47 


46 51 43 


52 53 50 


4 


65 63 58 


49 50 47 


47 47 43 


52 52 51 


5 


65 66 60 


51 54 49 


49 50 45 


52 50 51 


6 


65 63 62 


51 53 52 


48 49 47 


50 50 51 


7 


66 69 63 


53 56 51 


51 52 47 


50 50 50 


8 


65 73 62 


52 58 49 


49 55 46 


48 48 48 


9 


70 66 64 


58 54 51 


54 52 45 


48 49 48 


10 


66 57 61 


55 47 50 


51 45 46 


48 48 49 


11 


63 62 63 


53 54 49 


50 51 44 


49 40 49 


12 






48 44 40 


50 47 50 




Program Expenditures only include state and local funds. Monies 
expended from any federal grants are not included. 



R - Reading L - Language M - Math 



*Peroentile Scores 

76 



Graduation Requirements* 






Bilingual Staff 


1984-85 


isfifrsa 


Minimum Credit Units as follows: 






Certified Staff 






4.0 English 






Bilingual Endorsements 


0 


1 


2.0 Laboratory Science 






Provisional Bilingual Endorsements 


0 


0 


3.0 Soda! Studies 






ESL Endorsements 


0 


9 


2.0 Mathematics 






Provisional ESL Endorsements 


0 


22 


1.0 Physical Education 






Without Endorsements 


1 


0 


1.0 Cultural Enrichment 






Non-ln$tructk>nal Staff 


0 


0 


1.0 Practical Arts 






Total Certified Staff 


1 


32 


8.0 Electives 






Paraprofessional Staff 












Associate Degree 


1 


0 


22.0 TOTAL 






High School Dtptoma^GED 


0 


12 








Without High School Diploma^GED 


0 


0 


•Graduation Requirements effective for class of 1989 




Non-ln$tructk>nal Staff 


0 


0 








Total Paraprofessional Staff 


1 


12 








TOTAL 


2 


44 


Graduates and Dropout* 1988-89 














High School 


Dropout 










Graduates 


Bate 


Vocational Education Programs 


Students Served 


ABC Unified 


1,654 


5.26% 






1989-90 


Maricopa County 


17,467 


9.76% 


Accounting 




311 


Arizona 


31,423 


10.17% 


Business/General Office 




1,552 








Marketing 




89 








Radio/TV Production 




33 


Absentee Rate 1988-89 






Cosmetology 




20 


ABC. 


Maricooa Co. 


Arizona 


Health Care 




15 


Grades K-8 5.30% 


5.65% 


5.77% 


Life Management Education 




1,084 


Grades 9-12 4.30% 


6.33% 


6.62% 


Industrial Arts 




836 








Automotive 




87 




* 




Drafting, Graphic Arts, Commercial Arts 




37 


Percent Limited English Proficient 


(LEP) 1988-89 




Welding 




1 


ABC. 


Maricopa Co. 


Arizona 


Career Assessment and Services 




1,066 


Grades K-8 2.48% 


6.45% 


2.26% 


Hospitality 




8 


Grades 9- 12 1.19% 


2.10% 


2.72% 


Law Enforcement 




6 








Machine Shop 




2 








TOTAL 




5,147 


Students Scoring Below the 40th Percentile 1988-89* 










ABC. 


Maricopa Co. 


Arizona 








Grades K-8 17.65% 


35.98% 


40.55% 








Grades 9-12 22.88% 


37.30% 


41.24% 








* Standardize Achievement Test 













PROGRAMS AND SERVICES 




Bilingual Programs 


1984-85 


1988-89 


K-6 Transitional Bilingual 


0 


0 


7-12 Secondary Bilingual 


0 


0 


K-1 2 Bilingual Bicultural 


0 


0 


English as a Second Language (ESL) 


0 


364 


Individual Education 


0 


44 


PHLOTE* Students 


563 


803 


New and Continuing LEP Students 


97 


415 



* Primary Home Language Other Than English 



O 




77 



APPENDIX C 



California Department of Education 

District Performance Report Summary, 1989-90 



District: EPIC UNIFIED 

County: SAN DIABLO 

CD code: 60-54321 This report is based on 1 ,507 seniors. 

1-Year 



Quality Indicator 


1989-90 


Target 


Growth 2 
from 


Growth 2 
from Base 5 


1989-90 

Relative 


Performance Levels 


Percent 


Met’ 


1988-89 


1987-88 


Rank 4 


CAP Achievement 


Reading - commendable & above 


28 


• 


0 


2 


53 


Reading - adequate & above 


71 




0 


1 


64 


Mathematics - commendable & above 


39 


• 


3 


6 


81 


Mathematics - adequate & above 


77 


* 


-1 


4 


75 


Direct Writing - commendable & above 


27 


• 


3 


3 


83 


Direct Writing - adequate & above 


60 




3 


3 


87 


Curriculum 


Geometry completion 


64.3 


• 


0.7 


5.6 


69 


Four or more years of English 


75.5 




0.6 


-5.1 


52 


a-f course enrollments 


47.0 




8.1 


-1.0 


49 


Dropout Complement 

(100 minus % dropping out) 


Three-year derived rate 5 


94.4 


* 


1.5 


-1.2 


77 


College Bound 


a-f course completions 


40.3 




5.9 


9.3 


79 


Four-year college attendance 


21.4 


• 


0.5 


2.8 


79 


SAT verbal - at least 450 


16.9 




-0.4 


0.6 


49 


SAT mathematics - at least 500 


21.7 




0.5 


0.9 


66 


Advanced placement - 3 or better 


13.3 


• 


0.6 


7.3 


54 


Average Performance Value 


54.4 


• 


1.6 


1.9 


81 



Summary 


1990 Average 
Performance 


Target 


Percent 
9 Change 


Percent 

Change 


Relative 




Value 


Met' 


from 1988-89 


from Base 


Rank 


District values 


54.4 


• 


3.0 


3.6 


81 


State values 


48.9 




1 .9 


3.4 


N/A 


Percent change is the increase in the pool of students who met performance levels. 






Average performance value results for small districts (30 or fewer seniors) are not calculated because they tend to be unstable 
from year to year. Values based on fewer than 1 00 seniors should be interpreted with caution if being used for evaluation of 
district performance and program modification 

* Stars (*) indicate performance or base-year growth targets were met for the quality indicators and the 1990 average 
performance value. Refer to the “How to Read" section in the Interpretive Guide for details. 

2 

Growth is the change in the percent of students meeting performance levels. Each positive percentage point indicates 
that one percent more students met performance levels than In the past. 

3 The base year for all indicators is 1987-88 except CAP direct writing (1988-89), dropout complement, SAT, and AP (1986-87). 

^ The relative rank is the weighted average of schools* relative ranks; schools are weighted by size 

^ The three-year derived dropout complement is weighted by four in calculating the Average Performance Value. 

Missing values. Refer to The Report" section of the Interpretive Guide for details on value substitution. 
a All schools in the district had missing values in 1989-90. b 1988-89 missing c 1987-88 missing 

Excerpt from; California High School Performance Report Summary. 1990. ^arrampntn PA 
California Department of E ducat i on. Program evaluation and BSiiiFcF DlC ision ' ^ 
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